Skip to content

feat(telemetry): add credential.fetch tracing across all providers#12

Open
stoader wants to merge 1 commit into
mainfrom
otel
Open

feat(telemetry): add credential.fetch tracing across all providers#12
stoader wants to merge 1 commit into
mainfrom
otel

Conversation

@stoader

@stoader stoader commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator

Instrument every credential provider (aws, azure, gcp, generic, githubapp, k8ssecret, oauth2ac, oauth2cc, oci, rfc7523, rfc8693, vault) with OTel spans describing each credential fetch/refresh, plus nested exchange spans where a fetch involves a sub-step (gcp STS exchange + IAM impersonation, vault's GCP/Azure worker exchanges linked back to the fetch that started them).

  • New shared pkg/telemetry package: Tracer() (tp -> ambient span's provider ->
    global fallback), IdentityTokenAttrs() (subject/audience/iss/expiry from an
    unverified JWT, for telemetry only), RecordResult().
  • New shared pkg/util.ContextHolder: atomically stores a context.Context to smuggle a span-bearing context through third-party callback interfaces that don't accept one (aws stscreds.IdentityTokenRetriever, oauth2.TokenSource).
  • WithTracerProvider option added to every package, wired through to the daemon's telemetry provider.
  • A correlation_id, generated once per provider instance, ties together every fetch/refresh emitted by that instance.
  • rfc7523/rfc8693's additionalFields (an operator-configurable, otherwise free-form map) are only surfaced in telemetry via an explicit allowlist of the known-safe identifier keys populated by the Anthropic/OpenAI WIF integrations, rather than spread wholesale, to avoid an arbitrary operator-supplied value ending up in exported spans.
  • vault's credential.fetch span now includes cfg.vault_addr.

Instrument every credential provider (aws, azure, gcp, generic, githubapp,
k8ssecret, oauth2ac, oauth2cc, oci, rfc7523, rfc8693, vault) with OTel spans
describing each credential fetch/refresh, plus nested exchange spans where a
fetch involves a sub-step (gcp STS exchange + IAM impersonation, vault's
GCP/Azure worker exchanges linked back to the fetch that started them).

- New shared pkg/telemetry package: Tracer() (tp -> ambient span's provider ->
  global fallback), IdentityTokenAttrs() (subject/audience/iss/expiry from an
  unverified JWT, for telemetry only), RecordResult().
- New shared pkg/util.ContextHolder: atomically stores a context.Context to
  smuggle a span-bearing context through third-party callback interfaces that
  don't accept one (aws stscreds.IdentityTokenRetriever, oauth2.TokenSource).
- WithTracerProvider option added to every package, wired through to the
  daemon's telemetry provider.
- A correlation_id, generated once per provider instance, ties together every
  fetch/refresh emitted by that instance.
- rfc7523/rfc8693's additionalFields (an operator-configurable, otherwise
  free-form map) are only surfaced in telemetry via an explicit allowlist of
  the known-safe identifier keys populated by the Anthropic/OpenAI WIF
  integrations, rather than spread wholesale, to avoid an arbitrary
  operator-supplied value ending up in exported spans.
- vault's credential.fetch span now includes cfg.vault_addr.
@stoader
stoader requested review from a team, rappizs and waynz0r and removed request for rappizs and waynz0r July 20, 2026 14:38
@bonifaido
bonifaido requested a review from Copilot July 23, 2026 08:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds OpenTelemetry tracing across tokenex credential providers so credential fetch/refresh operations emit credential.fetch spans (and linked sub-spans for multi-step exchanges), with consistent config/result attributes and optional injection of a consumer-supplied TracerProvider.

Changes:

  • Introduces shared pkg/telemetry helpers (Tracer, IdentityTokenAttrs, RecordResult) and pkg/util.ContextHolder for propagating span-bearing contexts through callback-style interfaces.
  • Instruments multiple providers (AWS/Azure/GCP/Vault/OAuth2/etc.) with fetch/exchange spans, correlation IDs, and standardized attributes.
  • Adds WithTracerProvider(...) options across providers and updates module dependencies / Go version metadata.

Reviewed changes

Copilot reviewed 44 out of 45 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pkg/vault/telemetry.go Adds Vault fetch span constants and attribute helpers.
pkg/vault/option.go Adds WithTracerProvider option for Vault provider.
pkg/vault/gcp.go Adds linked exchange span for Vault→GCP access token exchange and propagates tracer/span context.
pkg/vault/creds.go Wraps Vault secret retrieval in credential.fetch span and links worker exchange spans back to fetch span.
pkg/vault/azure.go Adds linked exchange span for Vault→Azure access token exchange and propagates tracer/span context.
pkg/util/context_holder.go Adds atomic context “smuggling” helper for callback interfaces without context parameters.
pkg/telemetry/telemetry.go Adds shared OTel tracer selection + JWT identity token attribute extraction + unified span result recording.
pkg/rfc8693/telemetry.go Adds RFC8693 fetch span constants and attribute helpers (with allowlisted additionalFields).
pkg/rfc8693/rfc8693.go Wraps RFC8693 token exchange in credential.fetch span and records token/identity attrs.
pkg/rfc8693/option.go Adds WithTracerProvider option for RFC8693 provider.
pkg/rfc7523/telemetry.go Adds RFC7523 fetch span constants and attribute helpers (with allowlisted additionalFields).
pkg/rfc7523/rfc7523.go Wraps RFC7523 exchange in credential.fetch span and records token/identity attrs.
pkg/rfc7523/option.go Adds WithTracerProvider option for RFC7523 provider.
pkg/oci/telemetry.go Adds OCI fetch span constants and attribute helpers.
pkg/oci/option.go Adds WithTracerProvider option for OCI provider.
pkg/oci/creds.go Wraps OCI token exchange in credential.fetch span and records token/identity attrs.
pkg/oauth2cc/telemetry.go Adds OAuth2 client-credentials fetch span constants and attribute helpers.
pkg/oauth2cc/option.go Adds WithTracerProvider option for OAuth2CC provider.
pkg/oauth2cc/oauth2cc.go Wraps OAuth2 client-credentials token retrieval in credential.fetch spans with correlation.
pkg/oauth2ac/telemetry.go Adds OAuth2 auth-code/refresh telemetry attribute helpers.
pkg/oauth2ac/options.go Adds WithTracerProvider option for OAuth2AC provider.
pkg/oauth2ac/oauth2ac.go Adds spans around auth-code exchange and refresh flows, linking refresh spans to prior authorization.
pkg/k8ssecret/telemetry.go Adds k8s secret fetch/remove span constants and attribute helpers.
pkg/k8ssecret/option.go Adds WithTracerProvider option for k8ssecret provider.
pkg/k8ssecret/k8ssecret.go Adds spans for Secret add/update/delete events and initial validation publish path.
pkg/githubapp/telemetry.go Adds GitHub App fetch span constants and attribute helpers.
pkg/githubapp/option.go Adds WithTracerProvider option for GitHub App provider.
pkg/githubapp/githubapp.go Wraps installation token minting in credential.fetch spans.
pkg/generic/telemetry.go Adds generic provider fetch span constants and attribute helpers.
pkg/generic/option.go Adds WithTracerProvider option for generic provider.
pkg/generic/creds.go Wraps generic provider token creation in credential.fetch spans and records identity attrs.
pkg/gcp/telemetry.go Adds GCP fetch span constants and attribute helpers.
pkg/gcp/sts.go Adds STS exchange span and context propagation for callback-based token sourcing.
pkg/gcp/option.go Adds WithTracerProvider option for GCP provider.
pkg/gcp/iam.go Adds IAM impersonation span and records impersonated token expiry attrs.
pkg/gcp/creds.go Wraps GCP access token generation in fetch spans and wires STS/IAM subspans.
pkg/azure/telemetry.go Adds Azure fetch span constants and attribute helpers.
pkg/azure/option.go Adds WithTracerProvider option for Azure provider.
pkg/azure/creds.go Wraps Azure assertion-based token retrieval in fetch spans and records identity attrs in callback.
pkg/aws/token_retriever.go Uses ContextHolder to propagate span context into AWS identity token retriever callback.
pkg/aws/telemetry.go Adds AWS fetch span constants and attribute helpers.
pkg/aws/option.go Adds WithTracerProvider option for AWS provider.
pkg/aws/creds.go Wraps AWS credential retrieval in fetch spans and propagates context to IdentityTokenRetriever callback.
go.mod Updates Go version metadata and dependency requirements (including OTel and uuid).
go.sum Updates dependency checksums to match updated requirements.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/gcp/sts.go
Comment on lines +68 to 72
tok := &oauth2.Token{
AccessToken: resp.AccessToken,
TokenType: "Bearer",
ExpiresIn: resp.ExpiresIn,
Expiry: time.Now().Add(time.Duration(resp.ExpiresIn) * time.Second),
Comment thread pkg/oauth2cc/telemetry.go
Comment on lines +38 to +43
func fetchSpanResultAttrs(tok *oauth2.Token) []attribute.KeyValue {
return []attribute.KeyValue{
attribute.Bool("credential.expires", true),
attribute.String("credential.expires_at", tok.Expiry.UTC().Format(time.RFC3339)),
}
}
Comment thread pkg/aws/creds.go
Comment on lines +85 to +86
// we need to store current ctx as provider.Retrieve(fetchCtx) invoked bellow
// doesn't pass the fetch to TokenRetriever.GetIdentityToken()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants